<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Histogram - 4 : Histogram Backprojection</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d6/d00/tutorial_py_root.html">OpenCV-Python Tutorials</a></li><li class="navelem"><a class="el" href="../../d2/d96/tutorial_py_table_of_contents_imgproc.html">Image Processing in OpenCV</a></li><li class="navelem"><a class="el" href="../../de/db2/tutorial_py_table_of_contents_histograms.html">Histograms in OpenCV</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Histogram - 4 : Histogram Backprojection </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>In this chapter, we will learn about histogram backprojection.</p>
<h2>Theory </h2>
<p>It was proposed by <b>Michael J. Swain , Dana H. Ballard</b> in their paper <b>Indexing via color histograms</b>.</p>
<p><b>What is it actually in simple words?</b> It is used for image segmentation or finding objects of interest in an image. In simple words, it creates an image of the same size (but single channel) as that of our input image, where each pixel corresponds to the probability of that pixel belonging to our object. In more simpler words, the output image will have our object of interest in more white compared to remaining part. Well, that is an intuitive explanation. (I can't make it more simpler). Histogram Backprojection is used with camshift algorithm etc.</p>
<p><b>How do we do it ?</b> We create a histogram of an image containing our object of interest (in our case, the ground, leaving player and other things). The object should fill the image as far as possible for better results. And a color histogram is preferred over grayscale histogram, because color of the object is a better way to define the object than its grayscale intensity. We then "back-project" this histogram over our test image where we need to find the object, ie in other words, we calculate the probability of every pixel belonging to the ground and show it. The resulting output on proper thresholding gives us the ground alone.</p>
<h2>Algorithm in Numpy </h2>
<ol type="1">
<li>First we need to calculate the color histogram of both the object we need to find (let it be 'M') and the image where we are going to search (let it be 'I'). <div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cvfrom matplotlib <span class="keyword">import</span> pyplot <span class="keyword">as</span> plt</div><div class="line"></div><div class="line"><span class="comment">#roi is the object or region of object we need to find</span></div><div class="line">roi = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;rose_red.png&#39;</span>)</div><div class="line">hsv = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(roi,cv.COLOR_BGR2HSV)</div><div class="line"></div><div class="line"><span class="comment">#target is the image we search in</span></div><div class="line">target = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;rose.png&#39;</span>)</div><div class="line">hsvt = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(target,cv.COLOR_BGR2HSV)</div><div class="line"></div><div class="line"><span class="comment"># Find the histograms using calcHist. Can be done with np.histogram2d also</span></div><div class="line">M = <a class="code" href="../../d6/dc7/group__imgproc__hist.html#ga6ca1876785483836f72a77ced8ea759a">cv.calcHist</a>([hsv],[0, 1], <span class="keywordtype">None</span>, [180, 256], [0, 180, 0, 256] )</div><div class="line">I = <a class="code" href="../../d6/dc7/group__imgproc__hist.html#ga6ca1876785483836f72a77ced8ea759a">cv.calcHist</a>([hsvt],[0, 1], <span class="keywordtype">None</span>, [180, 256], [0, 180, 0, 256] )</div></div><!-- fragment --></li>
<li>Find the ratio \(R = \frac{M}{I}\). Then backproject R, ie use R as palette and create a new image with every pixel as its corresponding probability of being target. ie B(x,y) = R[h(x,y),s(x,y)] where h is hue and s is saturation of the pixel at (x,y). After that apply the condition \(B(x,y) = min[B(x,y), 1]\). <div class="fragment"><div class="line">h,s,v = <a class="code" href="../../d2/de8/group__core__array.html#ga8027f9deee1e42716be8039e5863fbd9">cv.split</a>(hsvt)</div><div class="line">B = R[h.ravel(),s.ravel()]</div><div class="line">B = np.minimum(B,1)</div><div class="line">B = B.reshape(hsvt.shape[:2])</div></div><!-- fragment --></li>
<li>Now apply a convolution with a circular disc, \(B = D \ast B\), where D is the disc kernel. <div class="fragment"><div class="line">disc = <a class="code" href="../../d4/d86/group__imgproc__filter.html#gac342a1bb6eabf6f55c803b09268e36dc">cv.getStructuringElement</a>(cv.MORPH_ELLIPSE,(5,5))</div><div class="line"><a class="code" href="../../d4/d86/group__imgproc__filter.html#ga27c049795ce870216ddfb366086b5a04">cv.filter2D</a>(B,-1,disc,B)</div><div class="line">B = np.uint8(B)</div><div class="line"><a class="code" href="../../d2/de8/group__core__array.html#ga7bcf47a1df78cf575162e0aed44960cb">cv.normalize</a>(B,B,0,255,cv.NORM_MINMAX)</div></div><!-- fragment --></li>
<li>Now the location of maximum intensity gives us the location of object. If we are expecting a region in the image, thresholding for a suitable value gives a nice result. <div class="fragment"><div class="line">ret,thresh = <a class="code" href="../../d7/d1b/group__imgproc__misc.html#gae8a4a146d1ca78c626a53577199e9c57">cv.threshold</a>(B,50,255,0)</div></div><!-- fragment --> That's it !!</li>
</ol>
<h2>Backprojection in OpenCV </h2>
<p>OpenCV provides an inbuilt function <b><a class="el" href="../../d6/dc7/group__imgproc__hist.html#ga3a0af640716b456c3d14af8aee12e3ca" title="Calculates the back projection of a histogram. ">cv.calcBackProject()</a></b>. Its parameters are almost same as the <b><a class="el" href="../../d6/dc7/group__imgproc__hist.html#ga4b2b5fd75503ff9e6844cc4dcdaed35d" title="Calculates a histogram of a set of arrays. ">cv.calcHist()</a></b> function. One of its parameter is histogram which is histogram of the object and we have to find it. Also, the object histogram should be normalized before passing on to the backproject function. It returns the probability image. Then we convolve the image with a disc kernel and apply threshold. Below is my code and output : </p><div class="fragment"><div class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</div><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"></div><div class="line">roi = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;rose_red.png&#39;</span>)</div><div class="line">hsv = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(roi,cv.COLOR_BGR2HSV)</div><div class="line"></div><div class="line">target = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;rose.png&#39;</span>)</div><div class="line">hsvt = <a class="code" href="../../d8/d01/group__imgproc__color__conversions.html#ga397ae87e1288a81d2363b61574eb8cab">cv.cvtColor</a>(target,cv.COLOR_BGR2HSV)</div><div class="line"></div><div class="line"><span class="comment"># calculating object histogram</span></div><div class="line">roihist = <a class="code" href="../../d6/dc7/group__imgproc__hist.html#ga6ca1876785483836f72a77ced8ea759a">cv.calcHist</a>([hsv],[0, 1], <span class="keywordtype">None</span>, [180, 256], [0, 180, 0, 256] )</div><div class="line"></div><div class="line"><span class="comment"># normalize histogram and apply backprojection</span></div><div class="line"><a class="code" href="../../d2/de8/group__core__array.html#ga7bcf47a1df78cf575162e0aed44960cb">cv.normalize</a>(roihist,roihist,0,255,cv.NORM_MINMAX)</div><div class="line">dst = <a class="code" href="../../d6/dc7/group__imgproc__hist.html#gab644bc90e7475cc047aa1b25dbcbd8df">cv.calcBackProject</a>([hsvt],[0,1],roihist,[0,180,0,256],1)</div><div class="line"></div><div class="line"><span class="comment"># Now convolute with circular disc</span></div><div class="line">disc = <a class="code" href="../../d4/d86/group__imgproc__filter.html#gac342a1bb6eabf6f55c803b09268e36dc">cv.getStructuringElement</a>(cv.MORPH_ELLIPSE,(5,5))</div><div class="line"><a class="code" href="../../d4/d86/group__imgproc__filter.html#ga27c049795ce870216ddfb366086b5a04">cv.filter2D</a>(dst,-1,disc,dst)</div><div class="line"></div><div class="line"><span class="comment"># threshold and binary AND</span></div><div class="line">ret,thresh = <a class="code" href="../../d7/d1b/group__imgproc__misc.html#gae8a4a146d1ca78c626a53577199e9c57">cv.threshold</a>(dst,50,255,0)</div><div class="line">thresh = <a class="code" href="../../d2/de8/group__core__array.html#ga61f2f2bde4a0a0154b2333ea504fab1d">cv.merge</a>((thresh,thresh,thresh))</div><div class="line">res = <a class="code" href="../../d2/de8/group__core__array.html#ga60b4d04b251ba5eb1392c34425497e14">cv.bitwise_and</a>(target,thresh)</div><div class="line"></div><div class="line">res = np.vstack((target,thresh,res))</div><div class="line"><a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv.imwrite</a>(<span class="stringliteral">&#39;res.jpg&#39;</span>,res)</div></div><!-- fragment --><p> Below is one example I worked with. I used the region inside blue rectangle as sample object and I wanted to extract the full ground.</p>
<div class="image">
<img src="../../backproject_opencv.jpg" alt="backproject_opencv.jpg"/>
<div class="caption">
image</div></div>
 <h2>Additional Resources </h2>
<ol type="1">
<li>"Indexing via color histograms", Swain, Michael J. , Third international conference on computer vision,1990.</li>
</ol>
<h2>Exercises </h2>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:37 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
